fix(datasource): persist runtime datasources across restart#2096
Merged
Conversation
UI-created (runtime) datasources vanished on a node restart: datasource-admin persists via MetadataManager.register(), which is in-memory only unless a writable `datasource:` loader is wired — and standalone `serve` wires none. (Runtime objects survive because the protocol writes them straight to sys_metadata; datasources never touched that durable path.) Fix, self-contained in the datasource-admin plugin (no change to global register() semantics, no protocol surgery): - On write, persist the record to the durable `sys_metadata` table via the data engine (same store + row shape objects use) in addition to the in-memory register; remove the row on delete. Only the opaque `external.credentialsRef` is stored — never credential cleartext. - On boot (`start()`, before pool rehydration) restore persisted runtime datasources from sys_metadata back into the registry, so `listDatasources()` and pool rebuild see them. Code-defined (artifact) datasources are unaffected. - Degrades gracefully to prior in-memory behavior when no data engine is present. Tests: +2 (persist→restart→restored; delete removes the durable row); service- datasource 65 green. Verified live: created a runtime datasource, restarted the backend, it survived (PERSISTED ACROSS RESTART). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 5 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
This was referenced Jun 21, 2026
Closed
xuyushun441-sys
added a commit
that referenced
this pull request
Jun 21, 2026
…tion (#2112) The README mentioned "REST routes under /api/v1/datasources" but didn't enumerate them, and predated the Studio integration. Add: - a REST routes section (lifecycle + drivers catalog + by-name test + GET :name detail + introspection/sync routes), with the credential-split note; - a Studio integration section: datasource is managed through the metadata-admin engine (Setup → Integrations → Datasources), runtime records persist to sys_metadata and rehydrate on restart. Reflects #2083 (drivers), #2085 (remote-tables/object-draft/by-name test), #2086 (GET :name), #2091 (Setup nav), #2096 (persistence). Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A UI-created (runtime) datasource vanished on backend restart. Root cause:
datasource-adminpersists viaMetadataManager.register(), which is in-memory only unless a writabledatasource:loader is wired — and standaloneservewires none. (Runtime objects survive because the protocol writes them straight tosys_metadata; datasources never used that durable path.)Fix
Self-contained in the
datasource-adminplugin — no change to globalregister()semantics, no protocol surgery:sys_metadatatable via the data engine (same store + row shape runtime objects use); remove the row on delete. Only the opaqueexternal.credentialsRefis stored — never credential cleartext.start(), before pool rehydration): restore persisted runtime datasources fromsys_metadataback into the registry, solistDatasources()+ pool rebuild see them. Code-defined (artifact) datasources are unaffected.Verification
service-datasource65/65 green.sys_metadata, restarted the backend, and it survived (PERSISTED ACROSS RESTART: True).Pairs with
The Studio datasource-via-metadata-admin-engine work (objectui #1846 + framework #2091).
🤖 Generated with Claude Code